CV-Convolutional and image filtering

Convolution

2D Convolutional Kernel $(r+1)\times (r+1)$ 原点在正中间

$$y[n,m]=\sum _{i=-r}^r\sum _{j=-r}^{r}w[i,j]\cdot x[n+i,m+j]$$

• $y[n,m]$  是输出图像的像素值；
• $x[n+i,m+j]$  是输入图像的像素值，表示在位置  $(n+i,m+j)$  的邻域像素；
• $w[i,j]$  是卷积核的权重；
• $r$  是卷积核的半径，卷积核大小为  $(2r+1)\times (2r+1)$ 。

Convolution Result

• Height

$$H_{output}=\frac{H_{input}-F_H+2P}{S}+1$$

• Width

$$W_{output}=\frac{W_{input}-F_W+2P}{S}+1$$

Example

If the input data is an image of size $32\times 32\times 3$ , using $10$ filters of size $5\times 5\times 3$ for convolution, with a stride of $1$ and padding of $2$, what will be the final size of the input?

So, the output size is $32\times 32\times 10$ . Through the convolution operation, the feature map can retain the same length and width as the original input.

Padding

Preserve Spatial Dimensions (Same Padding)

• Without padding, the output size of a convolution layer decreases after every operation. Padding helps preserve the original input size by compensating for the loss of border pixels during convolution.

For example:

• Input size =  $n\times n$
• Kernel size =  $k\times k$
• Without padding: Output size =  $(n-k+1)\times (n-k+1)$
• With “same” padding: Output size =  $n\times n$

Prevent Information Loss at Borders

Without padding, the convolution operation does not consider the edges and corners of an image as much as the center regions. Padding ensures the borders are included in the computation, retaining more information.

Enable Deeper Networks

By maintaining consistent feature map sizes throughout the layers, padding allows for the design of deeper networks without rapidly shrinking the feature map size.

Control Output Size

Padding can be adjusted to produce desired output dimensions for specific applications (e.g., “same” padding or “valid” padding).

Improve Symmetry for Feature Extraction

Padding ensures that the kernel interacts symmetrically with all regions of the input image, which improves the extraction of features near edges.

Sampling in 2D

Downsampling

Assume we have a matrix A with size $4\times 4$ :

$M_1$ with size $2\times 4$$M_2$ with size $4\times 2$

$A_{downsampled}=M_1\times A\times M_2$

UpSampling

Assume we have a matrix $A$ with size $2\times 2$:

$M_1$ with size $4\times 2$$M_2$ with size $2\times 4$

$A_{upsampling}=M_1\times A\times M_2$

Laplacian in 2D

Assume we have a 8×8 matrix (image) B:

$$B=\left[\begin{array}{cccccccc}{b}_{11}& b_{12}& b_{13}& b_{14}& b_{15}& b_{16}& b_{17}& b_{18}\\ b_{21}& b_{22}& b_{23}& b_{24}& b_{25}& b_{26}& b_{27}& b_{28}\\ b_{31}& b_{32}& b_{33}& b_{34}& b_{35}& b_{36}& b_{37}& b_{38}\\ b_{41}& b_{42}& b_{43}& b_{44}& b_{45}& b_{46}& b_{47}& b_{48}\\ b_{51}& b_{52}& b_{53}& b_{54}& b_{55}& b_{56}& b_{57}& b_{58}\\ b_{61}& b_{62}& b_{63}& b_{64}& b_{65}& b_{66}& b_{67}& b_{68}\\ b_{71}& b_{72}& b_{73}& b_{74}& b_{75}& b_{76}& b_{77}& b_{78}\\ b_{81}& b_{82}& b_{83}& b_{84}& b_{85}& b_{86}& b_{87}& b_{88}\end{array}\right]$$

The Laplacian filter in 2D (for edge detection) is commonly represented as:

$$L=\left[\begin{array}{ccc}0& 1& 0\\ 1& -4& 1\\ 0& 1& 0\end{array}\right]$$

If we use the matrix to represent the Laplacian, the matrix is:

$$M=\left[\begin{array}{cccccccc}-2& 1& 0& 0& 0& 0& 0& 0\\ 1& -2& 1& 0& 0& 0& 0& 0\\ 0& 1& -2& 1& 0& 0& 0& 0\\ 0& 0& 1& -2& 1& 0& 0& 0\\ 0& 0& 0& 1& -2& 1& 0& 0\\ 0& 0& 0& 0& 1& -2& 1& 0\\ 0& 0& 0& 0& 0& 1& -2& 1\\ 0& 0& 0& 0& 0& 0& 1& -2\end{array}\right]$$

And the result is:

$$B_{\text{Laplacian}}=M\ast B+B\ast M$$

Question

Part I: Convolution and image filtering

1. Comparing different filters?

2. Comparing different scales / size of filter?

3.Separability property of a filter / convolution?

4. Convolution and correlation? How

5. How to work on a kernel approximating a 1st, 2nd derivative?

6. What is Fourier Transform? What is the usage? How to calculate in 1D? 2D? Padding if necessary.

7. Convolution in image domain is equivalent to multiplication in frequency domain. Why? Verify?

8. How to obtain image pyramid? Gaussian, Laplacian, Steerable? Calculate? What are the usages / applications of them?